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Abstract 



A mechanism to generate fermion-mass hierarchy in SO(IO) Grand Unified Theories 
is considered. We find that the lopsided family structure, which is suitable to the Large 
angle MSW solution to the solar neutrino oscillation, is realized without introducing extra 
matter fields if the hierarchy originates from the wave-function profile in an extra dimen- 
sion. Unlike the Froggatt-Nielsen mechanism, the SO(IO) breaking effect may directly 
contribute to the source of the hierarchy, i.e., the bulk mass terms. It naturally explains 
the difference of the hierarchical patterns between the quark and the lepton sectors. We 
also find the possibility of the horizontal unification, in which three generations of the 
matter fields are unified to a 3 dimensional representation of an SU(2) gauge group. 
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1 Introduction 



Supersymmetric Grand Unified Theory (SUSY GUT) is one of tlie most promising can- 
didates wliicli describe tlie interactions among elementary particles in very high energy 
phenomena SUSY extension of the Standard Model provides us a solution to the 
hierarchy problem |2], and the unification of three gauge interactions in the Standard 
Model to GUT symmetries explains the charge quantization which is realized in nature 
[3]. Moreover, these two scenarios are non-trivially consistent with each other since SUSY 
predicts the coincidence of the gauge coupling constants at the high energy scale jl], which 
is a necessary condition to realize the Grand Unification. 

Among several candidates for the GUT gauge group such as SU(5), SO(IO), and Eg, 
SO (10) has particularly attractive features. One is the matter unification. All the quarks 
and leptons including right-handed neutrinos in each generation are unified to a single 
16 dimensional spinor representation field. Also, SO(IO) is the smallest simple GUT 
group which does not require a particular particle content to cancel the gauge anomaly. 
Yukawa unification is another interesting point. Because of the matter unification to 
16 representation fields, the Yukawa-interaction terms between the matter (16j) and the 
Higgs {10 h) fields in the superpotential are restricted to the form of W 3 XijldildjlOn- 
Thus, all the Yukawa matrices for up- and down-type quarks, the charged leptons, and 
neutrinos are the same since they all originate from the above superpotential. This is not 
necessarily a desired situation since the above relations result in the identical masses for 
the fermions in each family and no flavor mixings which are completely different from the 
observed structure. It is, however, possible to break the above relations naturally through 
the non-renormalizable interactions between the matter fields and 45h Higgs fields which 
break SO(IO) by its Vacuum Expectation Value (VEV) such as 16il6jl0//45/f , because 
the Yukawa interactions may feel the SO(IO) breaking effect jSHH]- 

However, when we take into account the fact that the Yukawa matrices have hierar- 
chical structures and they are quite different among the quark and lepton sectors, SO(IO) 
GUT has a difficulty in reproducing such structures. Many of the recent neutrino os- 
cillation experiments support the Large Mixing Angle MSW solution j7j (LMA) to the 
solar neutrino anomaly and the large mixing in the second and third generation neu- 
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trinos is suggested by atmospheric neutrino oscillation experiments [9] and confirmed by 
the K2K long baseline experiment ^J. On the other hand, all the mixing angles in the 
quark sector are small and masses have hierarchical structures. The magnitude of the 
hierarchy is enormous, e.g., the ratio of the up-quark mass to the top-quark mass 
rrit is approximately 10"^. Therefore, a mechanism to explain such a large hierarchy 
is necessary. The Froggatt-Nielsen (FN) mechanism is an elegant scenario to generate 
hierarchical structures in the Yukawa matrices jTTj, in which the hierarchy comes from 
the difference between the fundamental scale M and a U(1)fn symmetry breaking scale 
(•I'fn)- The Yukawa matrices fu, fa and /e, and the neutrino mass matrix rriiy are given 
in terms of a small parameter e ~ ($fn)/^ and the U(1)fn charges Q of the fermions 
by ~ ^ gQfe)+Q(dp^ fij ^ ^Q{h)+Q{e])^ ^^^^ ^ij ^ ^Qih)+Q{ij) ^ where i 

and j are the generation indices, q, u^, d'^, I, and e'^ represent the corresponding quarks 
and leptons. It is known that even if we impose the SU(5) GUT relations among the 
U(l) charges, i.e., Q{qi) = Q«) = Q{e'r) = Q(10,) and Q(d^) = Q{h) = Q{5i), we 
reproduce the hierarchical structures for quarks and leptons, and an appropriate neu- 
trino mass matrix accounting for the solar neutrino anomaly by LMA. An example of the 
charge assignment is Q{10i) = 3, (^(lOs) = 2, ^(lOs) = 0, ^(83) = 1, (5(82) = 0, and 
(5(5i) = with e ~ A ~ 0.22 |T21E]. This type of Yukawa matrices is called lopsided 
family structure which gives successful masses and mixing patterns especially for the large 
mixing angles in the lepton sector 112 13 El • However, in SO(IO) GUT, the above 
successful mechanism does not work in a simple way. The hierarchical structures in the 
fermion masses cannot be lopsided since the SO(IO) symmetry requires Q(lOj) = Q{5i). 
The situation does not change even if we introduce the non-renormalizable coupling to 
45 H to avoid the SO (10) relation in the fermion masses since the hierarchy is controlled by 
the charges of the U(1)fn symmetry, which is not related to the SO(IO) breaking. There 
have been attempts to avoid this problem. An often considered way is to introduce new 
matter multiplets of 10 dimensional representation and mix or flip d'^ and / in the 16 and 
those in the newly introduced 10 multiplets by the SO(IO) breaking effect ^3 El El- 
In the context of the six dimensional SO(IO) GUT [18j, it has been also proposed the 
introduction of extra matter fields where the hierarchy is a consequence of the volume 
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suppression factors 19^. In these scenario, df^ and / originate from the 10 representation 
fields other than 16, and thus the matter unification is spoiled. This is a generic problem 
in SO(IO) GUT. Therefore, in order to solve this problem, the strong connection between 
the origin of flavor hierarchy and the SO(IO) breaking is necessary. 

In this sense, the recently proposed mechanism to generate the flavor hierarchy based 
on the wave-function profiles in the extra dimension is noteworthy [201 1^ 1221 12ni 121] • 
In the simplest scenario, our world has an jZ^ compactified extra spacial dimension, 
and the bulk mass terms for the bulk superfields make them localize on the branes at the 
orbifold fixed points [221 12BJ- Consequently, the values of the wave functions at a brane 
may be suppressed by a factor of exp(— |m|7ri?), where m is the bulk mass and R is the 
radius of the extra dimension. The flavor hierarchy is obtained in a similar way as the FN 
scenario such like Z*-' ~ exp(— (|mj| + |mj|)7ri?). Hebecker and March- Russell considered 
the scenario in SU(5) GUT and showed that the hierarchy can be reproduced with a 
natural parameter sets i26|. Again, in SO(IO), it seems that the SO(IO) relation, z.e., the 
same mj for all the matter fields in each family, is too strong to reproduce the hierarchy. 
However, it is possible to break the SO(IO) relations since we have additional contributions 
to the bulk masses m's when SO (10) is broken by the VEV of the bulk adjoint field whose 
existence is ensured by SUSY in five dimension. The additional contributions are not 
universal to all the matter fields but proportional to the U(l)x (c SU(5)xU(l)x C 
SO(IO)) charges, so that they can change the wave-function profiles without preserving 
the SO(IO) relations. 

In this paper, we construct an SO(IO) SUSY GUT model in 5-dimensional space- 
time in which the correct fermion-mass patterns are realized by the above mentioned 
mechanism. The matter fields in each family are unified to a single 16 representation field. 
Our scenario is compatible with already proposed doublet-triplet splitting mechanisms 
such as Dimopoulos-Wilczek mechanism |2ZI, boundary condition [2111211; etc. We also 
discuss a possibility of the horizontal unification which unifies the three families to the 
single three dimensional representation of SU(2) group. 

This paper is organized as follows: In Section 2, we construct a model and explain 
the mechanism. The parameter sets to realize the suitable fermion-mass patterns are 
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discussed in Section 3. Section 4 is devoted to discussions and conclusions. 



2 Model 

In this section, we first review SUSY theory on the space-time x jZ^^ and show that 
the 5-dimensional wave function of zero mode has an exponential shape. This mechanism 
is a key ingredient of the fermion mass hierarchy. Then, we discuss the SO(IO) models in 
this setup. 

2.1 Zero mode wave function due to bulk mass term 

We consider the S^/Z2 compactified 5-dimensional SUSY gauge theory |^. For our 
purpose, it is convenient to write down the action in terms of superfields in 4-dimensional 
superspace [25J. The action for the U(l) gauge theory with a bulk hypermultiplet is given 
by 



ttR 

4, 



S = d X dy 



d^e ( -w"Wa + h.c. 1 + I d'^ei dyV - —{(f) + 



A " ' J ' J V ' V2 

+ j d^e {H'e^^^^H' + He-^^^^H) + j d^O [H^m + dy - V2gQ^)H + h.c. 



where and V are the field strength and vector superfield associated with the U(l) 
gauge group, and is the chiral superfield where is in its lowest components. The gauge 
transformation for the superfield is </) ^ + l/{V2g)dyA with transformation parameter 
A. Note that the gauge coupling constant g has a mass dimension of —1/2. The chiral 
superfields H and H'^ are the components of the hypermultiplet and their U(l) charges 
are Q and —Q, respectively^. To be invariant under the orbifold projection, the Z2 parity 
is assigned to be even for W^, V, and H, and odd for (p and H'^. Also, the bulk mass 
parameter m should be Z2 odd such that m = m sgn{y). 

By the Kaluza-Klein (KK) decomposition, we can find the existence of a zero mode 
^The superfields H and H'^ do not mean the Higgs fields here. 
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with a localized wave function. We expand the chiral superfields H and H'^ as follows: 

oo oo 

H{y) = J2 Hn{x)Uy) , H\y) = K{^)9n{y) • (2) 

n=0 n=0 

To obtain the normalized kinetic terms in the 4-dimensional effective action, the functions 

fn{y) and gn{y) are solutions of the following differential equations: 

d d 
dy dy 

with the normalization conditions: 

/ dy fn{y)fm{y) = dy gn{y)gm{y) = Snm ■ (4) 

Jo Jo 

The zero mode wave function with correct Z2 parity is easily found to be: 



foiy) = \l ^_lZm.R 90{y) = 0. (5) 

Thus the zero mode wave function /o localizes exponentially at y = (irR) for m > 
(m < 0). For m < and —2mTTR ^ 1, since the value of the wave function at y = 
is exponentially suppressed by a factor of /o(0) ~ A/2|m| e~'™''^^, the strength of the 
couplings between the bulk fields and the brane fields on the y = brane becomes weak. 
This effect can be the origin of the fermion mass hierarchy if the Higgs fields are confined 
on the brane at y = |22j . 

For the massive modes, the KK masses are given by 

< = m'+{^^\ (6) 
2.2 Gauge symmetry breaking 

As we can see in Eq.(^, the VEV of the superfield is an additional contribution to the 
bulk mass m, and as a consequence the wave-function profile of the zero mode is changed 
[221 123 Ell- Note that the additional contribution is proportional to the U(l) charge. We 
identify the U(l) gauge symmetry as a subgroup of the SO(IO) GUT gauge group and 
if as a matter superfield of the 16 dimensional representation of SO(IO), and then the 
different wave-function profiles are realized for the matter fields embedded in 16 since all 
the components do not have the same U(l) charges. 
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We consider the vacuum where the superfield acquires the VEV while the matter 
superfields H and H'^ do not. In order to preserve = 1 SUSY in the 4-dimensional 
effective theory (it ensures the configuration to be true vacuum), the VEVs of F- and 
D-terms must vanish. The vanishing F-term conditions are satisfied by assuming {H) = 
{H^) = 0, the D = condition is not the same as the 4-dimensional case but as follows 

1221 123 EH: 

= -D = dy{cP), (7) 

where we denote that is its scalar component. The contribution from the field is due 
to the 5-dimensional kinetic term, i.e., the second term in Eq.(^. The above equation 
is automatically satisfied if = const., however, as mentioned earlier, the field has 
odd parity under y —y transformation which means that the VEV should have the 
form of (0) = v^^"^ sgn(?/). Substituting this form in the above equation, we obtain the 
non-vanishing D-terms on the branes as follows: 

-D = 2v'/^6iy)-6iy-nR)) . (8) 

The brane localized D-terms in Eq.® have to be canceled by the brane localized 
terms in the action. The most economical way is to add the Fayet-Iliopoulos D-terms 
on both branes. However, it is not applicable in our context since we identify the U(l) 
symmetry as a subgroup of SO(IO)^. The second way is to add the brane fields charged 
under the U(l) symmetry. For example, if we assume the presence of chiral superfields 
5*0 with charge q and 5*,^ with charge —q which are localized on the branes at ?/ = and 
hR, respectively, the D-term is 

-D = dy{<P)- gq\ {So)my) + gq\ {S^)\'6{y - ttR) . (9) 

The condition D = has a solution with a flat direction: 

2v'/' = gq\{So)\' = gq\{S^)\' . (10) 

This is the usual Z^-flat direction in the 4-dimensional effective theory. In the vacuum 

with V ^ 0, the U(l) symmetry is broken by the VEVs of Sq and S.,^. This is an important 

^The D-term cancellation by the Fayet-Iliopoulos term is possible if the gauge group is broken explicitly 
to SU(5)xU(l) on the brane by the boundary conditions. 
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point when we apply this gauge symmetry breaking to the SO (10) GUT breaking. The 
fact that the U(l) symmetry is broken indicates that the U(l) symmetry should not be 
included in the Standard Model gauge group. Moreover, the U(l) symmetry must be 
orthogonal to the Standard Model gauge group, otherwise the VEV of induces the D- 
term corresponding to the Standard Model gauge group which cannot be canceled without 
their breaking. Therefore, the U(l) subgroup of SO(IO) is uniquely determined to be so 
called U(l)x symmetry, and thus SO(IO) is broken down to SU(5). As we see later, this 
U(l)x breaking yields the correct masses and mixing patterns of the fermions through the 
wave-function deformation as well as the Majorana masses for the right-handed neutrinos. 

2.3 SO(IO) model 

We construct an SO(IO) GUT model which includes the above setup. The matter fields 
are unified to three copies of 16 representation, 16i, I62, and I63 of SO(IO), and they 
and their charge conjugation fields 16^ can propagate into the bulk. Also, the gauge fields 
including a chiral superfield 45^ as the component in the action Eq. are in the bulk. 
The Higgs field IQh is introduced as a brane field (on the y = brane) in order to obtain 
the suppressed Yukawa coupling constants for the first and second generations. To break 
SO (10) GUT, we need 45// Higgs field which has VEV along the B — L direction in the 
Dimopoulos-Wilczek mechanism [2Zj, and the Higgs fields of IQh and IQh are necessary 
to reduce the rank of gauge group from five to four and give the Majorana masses to the 
right-handed neutrinos. We put these fields on the ?/ = brane, and 16^ and 16^ on the 
y = 7cR brane so that they can play the same roles as 5*0 and in the above mechanism. 
In this setup, the superpotential is given by 

W = 16'i{mi + dy - V2g454,)16i 



Ai,16a6,10H + ^(16a6//)i(16,16,/)i 



M 

+ %)K)(45//,16h,T6h,10h,---) 

+ 5(i/-7ri?)K(16^,T6'^) , (11) 

where i = 1,2,3 are the generation indices and M is a parameter with mass dimension 
one. The first term is the required form from the SUSY in five dimension. We take 
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a flavor basis where the bulk mass terms rrii are diagonalized. The Yukawa coupling 
constants Xij are dimensionless quantities and the interaction terms with Xij induce the 
Majorana masses for the right-handed neutrinos after the SO(IO) breaking. We omit the 
other possible contractions which are irrelevant for the low-energy effective theory. 

The potential Vq needs to have an appropriate form for the Dimopoulos-Wilczek mech- 
anism. According to the simplest model in Ref. it is required to exist another IO2H, 
an additional pair of I62H and I62H, and several singlet fields^. On the y = ttR brane, 
we can write the arbitrary potential for the 16^ and 16'^;^, and in general, there are a 
number of vacua where SO(IO) is broken. The Standard Model singlet components 
and Nh (g 16// and 16//), and N'j^ and iV^ (g 16'// and 16^) acquire VEVs after min- 
imizing the potential Vq and V^- Since these singlets are charged under U(l)x subgroup 
of SO(IO), there is a non-trivial condition from the D-flatness which is given by 

= -Dvnu = ^(y) + %x(l(iVH)r - \{Nh)\')] 

-6{y~nR)[v'/'-5gxmH)\'-\{K)\')] , (12) 

where gx is the coupling constant of the U(l)x gauge interaction normalized such that 
the charge of is —5, and f is the VEV of the 45^ field in the U(l)x direction. Along 
the fiat direction in Eq. lfT^ . we obtain the vacuum where f 7^ as seen in the previous 
subsection. 

Even with the potential there are massless modes because of the global SO(IO) 
symmetry in the y = nR sector. We assume the modes are stabilized by the SUSY 
breaking effect. The minimal set of the massless modes are a pair of 10 and 10 of SU(5) 
and a Standard Model singlet since the VEVs of A^^ and N'^ break SO(IO) to SU(5), 
although it requires additional Higgs fields. The massless modes will obtain the masses of 
the order of TeV. Alternatively, the singlet mode can be stabilized without SUSY breaking 
effect. When we gauge the U(l)' symmetry under which only 16^ and 16'// are charged 
such as 16^ : 1 and 16^ : — 1, the constraint from the D-fiat condition is given by 

= -D^iiy 

= S{y) [eo + v"/'] - S{y - nR) [-^^ + v'^' - g'{\{K)\' - , (13) 

■^It is difficult to forbid the dimension five operators which cause the proton decay in SO(IO) model with 
the Dimopoulos-Wilczek mechanism. The suppression of these operators is discussed in Ref. [HI I16L . 
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where and are the Fayet-Ihopoulos i^-terms of the U(l)' symmetry on the branes 
at y = and y = nR, respectively, f'^^^ is the VEV of the (p component associated 
with the U(l)' symmetry, and g' is the gauge couphng constant. If 7^ —(.-k, we obtain 
H^'h)? - \ {^'h)? 7^ 0, so that 7^ is assured by Eq.dEI). The singlet mode is eaten by 
the U(l)' gauge field in this case. 

In the vacuum with f 7^ 0, the wave functions for the matter fields lOj, 5j, and 
Ij are deformed in a different way because of the — \/2(7l6^45(^16i terms in the super- 
potential (jllj) . The bulk mass terms after the SO(IO) breaking is modified as rrii 
iTLi — \/2gxQxv'^^'^ -I where Qx is the U(l)x charges of 10 : —1, 5 : 3, and 1 : —5. This is 
the direct connection between the origin of the flavor hierarchy and the SO(IO) breaking. 
As we see in the next Section, this modification gives a realistic fermion-mass and mixing 
hierarchy. The violation of the unwanted SU(5) relation in the Yukawa matrices fd = fj 
is obtained from the effect of 16jl6jl0//(45//) operators jSllE]- 

We can also consider the 5-dimensional models where the gauge symmetry is broken 
by the boundary conditions j2Hl- First, we discuss the 5-dimensional models on the space- 
time X S^/{Z2 X Z'^), where the Z2 and Z'^ break SO(IO) down to SU(4) x SU(2) x 
SU(2) gauge symmetry and 4D N = 2 SUSY down to = 1, respectively. In this scenario, 
the SO(IO) gauge symmetry is explicitly broken down to SU(4) x SU(2) x SU(2) on the 
brane at y = |nilE3Eni- We do not have to worry about the doublet-triplet splitting 
problem since we can introduce the Higgs field as H : (1, 2, 2) representation field and 
thus there is no color-triplet Higgs from the beginning [29j. The SO (10) breaking Higgs 
fields 16h and 16h also need not to be the full multiplets and we assume the existence of 
: (4, 1, 2) and • (4, 1, 2), and their VEVs break the gauge symmetry down to 
the Standard Model gauge group. The matter fields are also decomposed to : (4, 2, 1) 
and : (4, 1, 2) which are required to originate from the different 16 dimensional fields 
in the bulk due to the Z2 projection. By SU(4) x SU(2) x SU(2) invariant but SO(IO) 
non- invariant interactions, we obtain the violation of the unrealistic SU(5) relation of 
fd = between the Yukawa matrix of the down-type quarks fd and that of the charged 
leptons fe- For example, the following SO(IO) non-invariant interaction terms do not give 
a contribution to fd but to fe after the Standard Model singlet components of and 
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acquire VEVs: 

where aij are the dimensionless couphng constants. Including these contributions, we do 
not have any relation between the down-type- quark and lepton Yukawa matrices. Since 
the contributions should be comparable to the SO(IO) invariant terms of ^ rH in order 
to reproduce the fermion masses, the parameter M is required to be the same scale as 
the VEV of Therefore, we take all the dimensionfuU parameters to be of the order 
of the GUT scale ~ 10^*^ GeV. 

In addition, we can discuss the scenario where the Z2 and Z'2 break 4D N = 2 SUSY 
down to = 1 and SO(IO) down to SU(5) x U(l)x, respectively. Because of the Z2 
projection, we introduce six copies of 16 fields in the bulk. In this model, lOj and (5j and 
Ij) come from the different 16's, and then, can have different bulk masses. So, similar 
to SU(5) models in Ref. one can generate the fermion-mass hierarchy by choosing a 
parameter set (2 = 1 — 6) for six 16 representation matter fields. The doublet-triplet 
splitting problem can be solved by Dimopoulos-Wilczek mechanism. Essentially speaking, 
this is an SU(5) model but not SO(IO) model, and then the matter unification is spoiled. 
Thus, we will not discuss it in this paper. 

3 Fermion masses 

We consider the fermion masses in this scenario with the assumption that all the com- 
ponents of the Yukawa matrices on the brane are of order unity. We also assume that 
neither SO (10) nor SU(5) relations among the Yukawa matrices maintain because of the 
symmetry breaking effects on the brane as discussed in Subsection 2.3. The hierarchi- 
cal structures in the observed fermion masses and mixings are realized by means of the 
wave-function profiles of the zero modes discussed in Subsection 2.1. The lopsided family 
structure is given through the effect of non-vanishing f = (45^) in the bulk. 

The observed hierarchy can be approximately expressed by the power of the Cabibbo 
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angle A ~ 0.22. The CKM matrix is given by 
/I A A3\ 

VcKM ~ -A 1 AM . (15) 





The ratios of the masses at the GUT scale [HZ] are given by 

niu : rric ■ rrit ^ : X"^ : 1 , (16) 

nifi : : mfe ~ A'' : A^ : 1 , (17) 

rrie : : m,- ~ A^ : A^ : 1 . (18) 

Also, the ratio of the neutrino masses in the second and third generations is estimated 

from the ratio of the Am^ of the solar and atmospheric neutrino oscillations. If we assume 
the hierarchical mass pattern, the ratio is given by 



^'"atm. 

where we used the LMA solution for the solar neutrino oscillation which requires nearly 
bimaximal mixing in the Maki-Nakagawa-Sakata (MNS) matrix |3Bj as follows: 

/ 1/V2 -1/V2 e \ 
Vmns ~ 1/2 1/2 -1/V2 , (20) 
V 1/2 1/2 1/V2 J 

where e is a small parameter. 

The Yukawa couplings in the effective theory are the products of the Yukawa couplings 
and the values of the wave functions on the brane at ?/ = (/o(0)). From Eq.(jHI), the 
/o(0)'s are given by 



y^^;^e-|aW|c (aM<0) 



VmP (af = 0) 

where the superscript r stands for the belonging SU(5) representations such as 10, 5, 
and 1, and i is the generation index. The parameter c is \M\7rR. The a^f^ parameters 
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are dimensionless quantities expressed in terms of /ij = mi/\M\ and k = gxv'^/'^ /\M\ as 
follows:^ 

af'^^ = Hi + k, a[^^ = Hi — 3k, a^'^^ = Hi + 5k . (22) 

The parameter k represents the effect of SO(IO) breaking and the SO(IO) relation of 
al^^^ = a\^^ = a\'^^ restores in the hmit of 0. This parameter plays an important 
role to reproduce the fermion-mass hierarchy. Hereafter, we take l/isl = 1 which can be 
done without loss of generality. Defining the dimensionless suppression factors n] = 
/o(0)*'*-'^V iy|M|, we obtain the formulae of the Yukawa matrices: 

f^ = X^r^'^4'\ /:r = A>P%f, Z!^ = A>®nf °) . (23) 

The matrices A„, A^, and Ae are the Yukawa matrices on the brane at y = and their 
components are of order unity. For neutrinos, the Yukawa matrix and the Majorana mass 
matrix for the right-handed neutrinos are given by 




Again X^, and M/j are the Yukawa matrix and the Majorana mass matrix {Mr = A(16//)VM) 
on the brane at ?/ = 0, respectively. Since the neutrino masses are given through the see- 
saw mechanism mj, = f^M^^f'^ jSH], the suppression factors of n'^'^ cancel in this formula 
and thus the neutrino mass matrix is expressed by 

<■ = (A.M^^AJ)^^- nfr^^l , (25) 

where V2 ~ 174 GeV is the VEV of the Higgs field which couples to the up-type quarks 
and neutrinos. 

The formulae in Eqs. ()23p and ()24|) are the same as those of the FN mechanism except 

the relations in Eq. ()22j) . The number of the free parameters is reduced to be four which is 

Hii H2, c, and k compared to the case of FN mechanism in which six charges (three Q(lOj) 

and three Q{5i)) and a parameter e = ($)/M are adjustable. Interestingly, we see in the 

following that the four parameters are sufficient to reproduce the Yukawa structures. 

''in the case where SO(IO) is broken down to SU(4) x SU(2) x SU(2) by the boundary condition 
as discussed in the previous section, the left-handed multiplet and right-handed one ^'^ may have 
different bulk masses mf and mf . However, as we see later, only the case with mf ~ mf gives the correct 
mass patterns, and this is suitable to the horizontal unification and the gauge coupling unification. 
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Our goal is to reproduce the lopsided family structures given as follows: 

/ A6 A3 \ / A^ A3 A3 \ / A2 A A 

fu-\ X' X' X' ], ~ /J ~ A3 A2 aM , oc A 1 1 | . (26) 
\A3A2l/ \A11/ VAll 

We can see that this form of Yukawa matrices reproduce the hierarchical structures in 
Eqs. fjl5|) - (j^m) when we take into account the 0(1) ambiguities. The large mixing in the 
1-2 generation of the neutrinos is realized when the determinant of the 2-3 submatrix in 
is 0(A), which is also consistent with the mass relation in Eq.()19|). As mentioned in 
Introduction, the form is suitable to the FN mechanism. Similarly, in our mechanism, the 
above structures are given when the suppression factors have the form of 

n;^°)~(A3,AM), nf)~(A,l,l). (27) 

The above type of suppression factors is obtained in the following way. First, we 
consider the nf'^'^ factors. The last component in n-''""'' of 0(1) is given if IO3 is localized 
on the y = brane. From Eq.()21|). this is achieved by 

41°) = ±1 + A; > , (28) 

where ± represent the undetermined sign of fi^. The second component in n-"*^"^ of O(A^) 
is, in contrast, given by the IO2 localization on the y = nR brane such that 

4^°)c = (/i2 + k)c ~ log A^ 3.0 . (29) 

The first component in n^i^^^ of 0(A3) determine the relative size of the ni parameter as 
follows: 

(aS^°^ - 4^°^)c = (/ii - /i2)c ~ log 1.5 . (30) 

O(A^) 

(5) (5) 
Next, the ni factors constrain the parameters. The second and last components in n- 

of 0(1) give 

= ±1 - 3A; > , ^ = ^2 - 3A; > . (31) 
The suppression of the first component in n\^^ is given by 

af ) = (/ii - 3k)c ~ log A 1.5 . (32) 
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Figure 1: The configurations of tlie matter fields before and after tlie SO(IO) breaking 
are sliown. 

From tlie above conditions, the sign of fi^ is determined to be /is = +1 and the kc 
parameter is approximately given by 



kc 



-0.75 . 



(33) 



Once we postulate the value of k or c, we can determine the values of fii and fi2 by Eqs. 
and (jnOJ. For example, for k ~ —0.75 and c ~ 1, we find fii ~ —3.75 and /i2 ~ —2.25. 
We schematically explain this mechanism in FigO Without SO(IO) breaking, lOj and 5j, 
of course, have the same profiles. However, once the SO(IO) breaking effect is turned on, 
the profiles are deformed in such a way that the Yukawa structures become the currently 
observed form. 

When we think about the origin of the bulk mass parameters mi, there is a possibility 
that the origin is the spontaneous breaking of an SU(2) horizontal symmetry^. In fact, 
the suppression of the n^^"* is not necessarily important. Even if n^^"* = 0(1), the Yukawa 
matrices are in a good shape if we take into account the 0(1) ambiguities in the Yukawa 
couplings on the brane. The neutrino masses and mixings are explained by the anarchy 
scenario jlUl- In that scenario, the form of the suppression factors is given by |4T] 

^(^°)^(A^AM), (1,1,1). 



n- 



(34) 



Although this type of suppression factors gives a small value ~ to the Cabibbo angle, 
we think that the observed large value of the Cabibbo angle is due to the 0(1) ambiguity. 
^Similarly, one can consider the U(l) flavor symmetry. 
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An interesting possibility arises in this case. We find a parameter set of ~ —1, c ~ 3, 
fii ~ —1, fi2 ~ 0, and /i3 = 1, wliicli gives above suppression factors. Witli tlie relation 
fii = —fi3 and fi2 = 0, there is a possibility that the origin of the bulk mass parameters 
rrii are the VEV of the bulk adjoint field of the SU(2) horizontal symmetry in which the 
three generations of the 16 representation matter fields are unified to 3 representation. 
As discussed in the previous section, the VEV of the bulk SU(2) adjoint field gives rise 
to the D-terms on the branes and they are canceled by the D-terms given by the VEVs 
of the localized fields in the fundamental representation on the branes. The VEVs break 
the SU(2) symmetry completely and the couplings to the matter fields may induce off 
diagonal components of the Yukawa couplings which are absent in the SU(2) symmetric 
limit. This possibility is a special feature in SO(IO) GUT and can not be realized in SU(5) 
GUT. The existence of the U(l)x breaking is essential to give the different bulk masses 
to 10 and 5 fields. 

Finally, we comment on the neutrino masses. If we assume that all the dimension- 
full parameters are of the order of 10^^ GeV, we find the mass of the heaviest neutrino 
in Eq.()25|) is one order of magnitude smaller than the favored value by the atmospheric 
neutrino data of ~ O.OSeV. However, this is not a serious problem because the 0(1) ambi- 
guities easily reduce the Majorana masses of the right-handed neutrinos and/or increase 
the Dirac neutrino Yukawa couplings. 

Although the right-handed neutrino masses do not affect the neutrino masses, they 
might be important from the viewpoint of leptogenesis In the scenario with Eq. ljHHjl . 
our predictions on the lightest right-handed neutrino masses are given by 

~ M}1 ~ IQ-^M ~ lO^GeV . (35) 

This value is compatible with the leptogenesis scenario ^3 |33]- Of particular interest 
is the case with the horizontal symmetry. With the values of ~ — 1 and c ~ 3, we 
obtain the right-handed neutrino masses ~ 1 — 10 GeV, ~ 100 — 1000 GeV, and 
~ 10^~^ GeV. Since the lightest right-handed neutrino does not enter the thermal 
equilibrium before the electroweak phase transition, the partial lepton asymmetry carried 
by the particles except for the right-handed neutrino is converted to the baryon number 
by the sphaleron process even if there is no net B — L asymmetry [Hj. 
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We assumed in the superpotential in Eq.(fTT|) that the right-handed neutrinos acquire 
masses from the VEVs of y = brane fields. Alternatively, we can change the source of 
their masses to the y = tcR brane. However, in this case, Eq.(j2SI) is not applied, and then 
it leads too small neutrino masses or non-suitable forms for the LMA solution. 

4 Discussions and Conclusions 

We consider the scenarios with the lopsided family structure in SO(IO) GUT by using the 
5-dimensional wave function profile. We can reproduce the fermion masses and mixings 
including the LMA solution without introducing extra matter fields. The key point is that 
we directly connect the SO(IO) breaking effect to the origin of fermion-mass hierarchy, 
which is difficult in the usual FN scenario. 

We used the bulk mass terms as a origin of the hierarchy. The SO(IO) breaking effect 
gives additional contributions to the bulk mass terms so that the Yukawa matrices do not 
obey the unrealistic SO(IO) relation /« ~ /d ~ /e ~ fu- Moreover, we find the possibility 
of SU(2) horizontal unification of three generations, and that is possible in SO (10) rather 
than SU(5) GUT. 

"SU(5) or SO(IO)?" is an interesting question. From the viewpoint of the doublet- 
triplet splitting problem, SO(IO) might have an advantage since it has an intrinsic possi- 
bility to realize the doublet-triplet splitting by the Dimopoulos-Wilczek mechanism, but 
it is not easy. Since we need to reduce the rank of gauge group, the Higgs sector becomes 
complicated. It is, therefore, difficult to judge from this viewpoint. The fermion-mass 
hierarchy is another interesting viewpoint. We can say that SU(5) is better than SO(IO) 
in this case. The SU(5) relation of fd ~ fj is approximately good and the hierarchy is 
compatible to the FN mechanism. On the other hand, SO(IO) symmetry is too strong 
and it is required to introduce extra-matter fields. However, the concept of the matter 
unification is too beautiful to be thrown away. In this sense, the wave-function localiza- 
tion is an attractive possibility which enables a matter unified SO (10) GUT to be realistic 
and opens up a further possibility of horizontal unification. 

Considering the SUSY breaking in this scenario is also interesting. Severe experimental 
limits of the processes with Flavor Changing Neutral Current (FCNC) and CP violation 
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restrict the sfermion mass matrices to be some specific forms. In our scenario, tliere is 
an interesting possibility if SUSY is broken on the y = tcR brane. Because of the wave- 
function profiles, only the first and second generation fields can feel the SUSY breaking 
effect strongly. This is the desired situation for the effective SUSY scenario |45^ in which 
the sfermions of the first and second generations have large soft masses of the order of 
10 TeV so as to avoid the constraints from the FCNC and CP violating processes, and 
the third generation sfermions are as light as O(IOO) GeV for naturalness. Consideration 
in this direction is interesting, although we need the mechanism to suppress the gaugino 
masses and to generate the /i-term. The gauge mediation 46J is another candidate where 
the 16'fj^ and 16^ fields on the y = nR brane may be the messenger fields. 
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